7.3. Costs

Where can agent cost grow?

A model agent has no fixed price per request the way a CRUD endpoint does. Cost is a function of how many tokens the loop consumes, and the loop decides that at runtime. The usual growth sources:

• Repeated model calls inside one loop/task — each tool round-trip is another full prompt.
• Large prompts from session history, tools, skills, or runbooks (search_runbooks alone returns whole markdown documents that then live on in history).
• Expensive judge/evaluation calls that run a second model over every answer.
• Idle Kubernetes control plane, node, disks, registry, bucket, and observability retention.
• Retries or failures that repeat non-idempotent/expensive work.

Latency and cost usually share the same cause: unnecessary loop steps or context. If you fix one you generally fix both, which is why the same telemetry backs both this page and 7.2. Monitoring.

How do you verify the token accounting locally?

Run this section before you read the rest of the page. Everything after it explains the numbers you are about to read off your own session.

Prove the mechanism, do not assume it. With the self-hosted stack up (mise run observability:up) and the agent launched with the documented OTLP environment (OTEL_EXPORTER_OTLP_ENDPOINT=http://127.0.0.1:4318):

1. Send one question through the agent (mise run run and ask What is the status of the checkout service?, or drive a turn through mise run a2a).
2. Read the counter through Prometheus — the same series the dashboard and the AgentTokenTelemetryMissing alert watch:

curl -fsS 'http://localhost:9090/api/v1/query?query=agentops_tokens_token_total' \
  | jq '.data.result'

1. Open that turn's trace in MLflow at http://localhost:5000 and read the span attributes agentops.tokens.session.total and agentops.cost.session.estimate (the estimate is 0 until you set the per-1k prices).
2. Turn that zero into a number. Both prices default to 0.0, so the shipped estimate on the local path is always 0.0 — which proves nothing about the arithmetic. Look up the published input and output rates for your selected model, convert per-million prices to per-1k if needed, and record the source and date. Restart the agent with those rates:

cd agents/python
AGENT_INPUT_PRICE_PER_1K=<published input rate> \
  AGENT_OUTPUT_PRICE_PER_1K=<published output rate> \
  mise run run

Ask the same question again and reopen the new trace: agentops.cost.session.estimate is now non-zero and equals agentops.tokens.session.input / 1000 × your input rate + agentops.tokens.session.output / 1000 × your output rate. Recompute it by hand once; if they disagree, check the exact trace, configured units, and token fields before diagnosing the accounting.

Local Ollama bills nothing, so that number is a stated assumption: what this exact turn would have cost on the provider whose rate you entered. It is a budgeting signal you can defend because you can name its source and date — never a billing figure.

1. Force the budget to trip: restart the agent with AGENT_MAX_TOKENS_PER_SESSION=1, then send two turns in one session. The second short-circuits with error_code="TOKEN_BUDGET_EXHAUSTED" and the refusal message quoted below, before any model call.

If step 2 stays empty while spans still flow, the accounting is broken — the exact condition AgentTokenTelemetryMissing fires on in 7.2. Monitoring.

Which request and rate bounds are implemented?

Now you have a number; here is what stops it growing. Two mechanisms bound cost today, and neither counts money.

The application caps how much reasoning one request may do, and the gateway caps how fast any client may hit each surface. That is defense in depth: they bound work, not spend. Qwen3 runs on learner-owned compute for the local path, so the local path has no per-token invoice at all.

• A2A model-call cap. One A2A request runs at most AGENT_A2A_MAX_LLM_CALLS model calls. It is a typed setting in config.py, a2a_max_llm_calls: int = Field(default=12, ge=1, le=100), applied in server.py as the runner's max_llm_calls. Exceeding it fails the turn back to the caller as an error, not a silent truncation.
• Gateway rate limits in infra/agentgateway/host/config.yaml. Each route gets a localRateLimit token bucket per gateway instance: a request allowance that refills on a fixed interval and returns HTTP 429 once drained. The model route on :4000 allows 30 requests per 60s, the MCP route on :3000 allows 120, and the A2A route on :3001 allows 60.

Where a single request meets each of those bounds:

flowchart TD
  Client[Client] -->|message/send| A2A["gateway :3001 A2A<br/>60 req / 60s → HTTP 429"]
  A2A --> Agent[agent turn]
  Agent -->|at most 12 model calls| Cap["a2a_max_llm_calls = 12<br/>config.py → error to caller"]
  Cap -->|each model call| LLM["gateway :4000 model<br/>30 req / 60s → HTTP 429"]
  Agent -->|each tool call| MCP["gateway :3000 MCP<br/>120 req / 60s → HTTP 429"]

These are safety rails, not a monetary budget. There is no shipped dollar-cost alert or billing export.

What does one request actually cost?

Start with numbers, not the formula. The rate below is illustrative, not a vendor quote.

Suppose one session's trace reports agentops.tokens.session.input = 1500 and agentops.tokens.session.output = 400, and an operator has set AGENT_INPUT_PRICE_PER_1K=0.20 and AGENT_OUTPUT_PRICE_PER_1K=0.60:

input  : 1500 / 1000 * 0.20 = 0.30
output :  400 / 1000 * 0.60 = 0.24
estimate = 0.54

That 0.54 is what estimate_cost writes to agentops.cost.session.estimate for that session — the arithmetic you reproduced by hand two sections ago with your own rates. By default both prices are 0.0 for every provider, so the same session estimates 0.0. The mechanism runs either way, but the number is only meaningful once an operator sets published rates and reconciles the total against the provider's own invoice. Treat it as a budgeting signal, never as billing truth.

Generalised, that arithmetic is the formula to reuse for any provider:

request_cost = input_tokens * input_price_per_token
             + output_tokens * output_price_per_token
             + provider-specific cached/reasoning/tool charges

Use observed input/output tokens from the selected provider/instrumentation and the provider price effective on the run date. Do not hard-code prices in the agent or infer token counts from character length. Local Ollama has no API price but consumes machine time, memory, energy, and operator capacity.

How do I attribute cost to a session or tool?

Tokens are attributed per session only, despite the heading. There is no per-tool cost breakdown anywhere in the code, because the model reports one usage figure per call, not one per tool. No callback splits a turn's tokens across the tools it invoked.

Before deployment, each ADK evaluation case also reports its own tool calls, model calls, and tokens (4.4. Evaluations). Those describe one scripted case, not a production session. To approximate per-component weight you must measure by ablation: hold a question constant, change one toolset or docstring, and diff the first turn's prompt_token_count. That method is described in 3.4. Memory.

The general problem is closing the loop between "how many tokens did this consume?" and "who pays for it?" without inventing a number. Do that at the boundary where the token count is authoritative — the model response — instead of estimating from character length later.

This repository does it in one app-wide policy plugin, governance.py, so token accounting cannot be missing from an agent someone added later:

APP_NAME = "agentops-agent"


class AgentOpsPolicyPlugin(BasePlugin):
    """Apply the course's model, tool, and error policy to every agent in the app."""

    def __init__(self, name: str = "agentops_policy") -> None:
        super().__init__(name=name)

    async def before_model_callback(
        self, *, callback_context: CallbackContext, llm_request: LlmRequest
    ) -> LlmResponse | None:
        """Budget, then bound the history, then redact what survives."""
        for guard in (enforce_token_budget, compact_history, redact_request_pii):
            response = guard(callback_context, llm_request)
            if response is not None:
                return response
        return None

    async def after_model_callback(
        self, *, callback_context: CallbackContext, llm_response: LlmResponse
    ) -> LlmResponse | None:
        """Attribute this turn's tokens, then redact the response."""
        for guard in (record_token_usage, redact_response_pii):
            replacement = guard(callback_context, llm_response)
            if replacement is not None:
                return replacement
        return None

    async def before_tool_callback(
        self, *, tool: BaseTool, tool_args: dict[str, Any], tool_context: ToolContext
    ) -> dict[str, Any] | None:
        """Reject malformed arguments to a mutating action before it touches state."""
        return validate_actions(tool, tool_args, tool_context)

    async def after_tool_callback(
        self, *, tool: BaseTool, tool_args: dict[str, Any], tool_context: ToolContext, result: dict[str, Any]
    ) -> dict[str, Any] | None:
        """Harden untrusted tool output and redact PII before the model sees it."""
        return secure_tool_output(tool, tool_args, tool_context, result)

    async def on_model_error_callback(
        self, *, callback_context: CallbackContext, llm_request: LlmRequest, error: Exception
    ) -> LlmResponse | None:
        """Turn a provider failure into an actionable response instead of a stack trace."""
        return handle_model_error(callback_context, llm_request, error)

    async def on_tool_error_callback(
        self, *, tool: BaseTool, tool_args: dict[str, Any], tool_context: ToolContext, error: Exception
    ) -> dict[str, Any] | None:
        """Return a stable, non-sensitive error for a failed tool."""
        return handle_tool_error(tool, tool_args, tool_context, error)


def build_app(selected_root: BaseAgent | Workflow) -> App:
    """Attach the complete cross-cutting policy to one executable application."""
    return App(name=APP_NAME, root_agent=selected_root, plugins=[AgentOpsPolicyPlugin()])

record_token_usage (the after_model_callback) in budget.py reads usage_metadata from each response. Input includes prompt and tool-result prompt tokens; output includes candidates and reasoning tokens. If a compatible provider reports only a total, the unclassified remainder goes into the output bucket so enforcement cannot silently count zero. The callback then emits the running totals as OpenTelemetry span attributes plus a counter:

    with _session_usage_lock(callback_context):
        input_tokens, output_tokens = session_usage(callback_context)
    used = input_tokens + output_tokens
    if used < settings.max_tokens_per_session:
        return None
    message = (
        f"This session has exhausted its token budget ({used} of {settings.max_tokens_per_session} tokens used). "
        "Start a new session to continue, or raise AGENT_MAX_TOKENS_PER_SESSION if the work warrants it."
    )
    return LlmResponse(
        content=types.Content(role="model", parts=[types.Part(text=message)]),
        error_code="TOKEN_BUDGET_EXHAUSTED",
        error_message="Per-session token budget exhausted.",
    )


def record_token_usage(callback_context: CallbackContext, llm_response: LlmResponse) -> None:
    """``after_model_callback``: attribute this turn's tokens to the session.

    Accumulates into session state and emits OTel span attributes (visible in
    MLflow traces) plus a metric counter (scraped by Prometheus). Returns
    ``None`` so the response continues to the next callback unchanged.
    """
    usage = llm_response.usage_metadata
    if usage is None:
        return
    turn_input = (usage.prompt_token_count or 0) + (usage.tool_use_prompt_token_count or 0)
    turn_output = (usage.candidates_token_count or 0) + (usage.thoughts_token_count or 0)
    classified_total = turn_input + turn_output
    # Some compatible providers report only a total. Keep the budget fail-closed
    # by assigning otherwise-unclassified usage to the output bucket rather than
    # silently recording zero tokens.
    turn_output += max((usage.total_token_count or classified_total) - classified_total, 0)
    with _session_usage_lock(callback_context):
        input_tokens, output_tokens = session_usage(callback_context)
        input_tokens += turn_input
        output_tokens += turn_output
        callback_context.state[INPUT_TOKENS_KEY] = input_tokens
        callback_context.state[OUTPUT_TOKENS_KEY] = output_tokens

    _TOKEN_COUNTER.add(turn_input, {"direction": "input"})
    _TOKEN_COUNTER.add(turn_output, {"direction": "output"})
    span = trace.get_current_span()
    span.set_attribute("agentops.tokens.session.input", input_tokens)
    span.set_attribute("agentops.tokens.session.output", output_tokens)
    span.set_attribute("agentops.tokens.session.total", input_tokens + output_tokens)
    span.set_attribute("agentops.cost.session.estimate", estimate_cost(input_tokens, output_tokens))

The callback lock is keyed by ADK's application, user, and session IDs. It prevents overlapping callbacks over the same in-memory state from losing an increment, and its registry entry disappears after the last waiter exits.

That callback lock cannot repair two detached session snapshots. The A2A server therefore uses _SessionSerializingRunner: it queues a user's overlapping turns for one session before ADK loads state, holds the lock until the event stream and compaction finish, then lets the next turn read the persisted total. Different users or sessions still run concurrently.

Both locks are process-local. The shipped single-replica runtime has one process and one SQLite writer; a multi-replica deployment needs atomic accounting or a distributed per-session gate in its shared session store. Direct Runner callers and ADK Web also sit outside the A2A invocation gate.

estimate_cost multiplies accumulated tokens by AGENT_INPUT_PRICE_PER_1K / AGENT_OUTPUT_PRICE_PER_1K, both of which default to 0.0. Until prices are configured, a zero estimate means unconfigured attribution. This applies to Gemini too; it is not evidence of free usage. Set your provider's published per-1k rates to make agentops.cost.session.estimate meaningful; no vendor price is hardcoded. The span attributes ride the MLflow trace; the agentops.tokens counter is what Prometheus scrapes (as agentops_tokens_token_total, tagged by direction).

Trace one model call end to end:

sequenceDiagram
  participant ADK as ADK runner
  participant B as enforce_token_budget (before_model)
  participant M as Model (Ollama / gateway)
  participant R as record_token_usage (after_model)
  participant S as Session state (persisted)
  participant O as OTel span + counter
  ADK->>B: before_model_callback
  alt budget already spent
    B-->>ADK: LlmResponse TOKEN_BUDGET_EXHAUSTED
  else budget left (or unset)
    B-->>ADK: None (continue)
    ADK->>M: model call
    M-->>ADK: LlmResponse with usage_metadata
    ADK->>R: after_model_callback
    R->>S: accumulate budget:input/output_tokens
    R->>O: span agentops.tokens/cost.session.* + agentops.tokens counter
  end

Diagram in words: ADK checks the session budget before the model call. If it is spent, ADK returns TOKEN_BUDGET_EXHAUSTED; otherwise the model runs and reports usage. The after-model callback adds those tokens to persisted session state, then sends the running token and cost attributes to OpenTelemetry.

How do I enforce a budget?

Attribution is measurement; enforcement is a decision to stop. The app plugin calls enforce_token_budget from before_model_callback, short-circuiting the next model call once the session's running total reaches AGENT_MAX_TOKENS_PER_SESSION. The caller gets an actionable message instead of a silent failure or an open-ended bill:

if used < settings.max_tokens_per_session:
    return None
message = (
    f"This session has exhausted its token budget ({used} of {settings.max_tokens_per_session} tokens used). "
    "Start a new session to continue, or raise AGENT_MAX_TOKENS_PER_SESSION if the work warrants it."
)
return LlmResponse(
    content=types.Content(role="model", parts=[types.Part(text=message)]),
    error_code="TOKEN_BUDGET_EXHAUSTED",
    error_message="Per-session token budget exhausted.",
)

The scope is a deliberate design decision. The session-state keys carry no temp: prefix:

# Session-state keys. No ``temp:`` prefix, so DatabaseSessionService persists
# the running totals across turns — the budget covers the whole conversation.
INPUT_TOKENS_KEY = "budget:input_tokens"
OUTPUT_TOKENS_KEY = "budget:output_tokens"

Without the temp: prefix, DatabaseSessionService persists the totals across turns. Two consequences follow:

• The budget is per conversation, not per turn: further model calls stop after tracked usage reaches the threshold.
• A new session starts the counter at zero. That is the intended reset, and it is also the easy bypass — a client that opens a fresh session per request never hits the threshold.

The budget bounds one conversation, not one client. It works identically on the free local path and the gateway path — it counts tokens, not dollars. Left unset (None), enforcement is disabled and only measurement runs.

Where does the token budget stop protecting you?

The mechanism has real edges. Know them before you rely on the number:

• Admitted calls can overshoot. The callback checks accumulated usage before a model call and records that call afterward. A call that starts below the threshold may finish above it; only the next call is refused. The shipped A2A server queues same-session turns, but direct Runner callers, ADK Web, or another replica can still admit overlapping calls. Neither process-local gate reserves tokens or sets a per-call output-token cap.
• A usage-less response defeats it. record_token_usage returns early when llm_response.usage_metadata is None, so a response with no reported usage adds nothing to the running total. The gateway streaming path reports no usage on streamed responses — one reason AGENT_A2A_STREAMING defaults off. Any provider or path that omits usage_metadata also silently escapes both the counter and the budget.
• A new session resets it. As above, the budget is per conversation; a client that rotates sessions is unbounded.

What does the dashboard measure today?

The shipped dashboard graphs request/error rate and latency. No panel shows tokens, and none shows dollars.

The cost signals that do exist:

• The agentops_tokens_token_total counter, by direction, added with token telemetry. It would support a token-throughput panel, but the shipped dashboard does not include one.
• The per-session token and cost-estimate attributes that traces carry, described above.
• The AgentTokenTelemetryMissing ticket alert, the one guard that ships for this mechanism. Spans flowing without any agentops_tokens_token_total increase means the budget callback or the metrics pipeline broke, and the dashboard would otherwise lie by omission.

A USD time series still needs a verified price source per model, so a monetary panel stays an explicit opt-in rather than a shipped claim.

What does the GKE lab cost per month?

The cloud lab is optional and billable. At prices checked on 31 July 2026, its starting fixed resources cost about USD 28.25 monthly when the billing account's GKE, disk, and registry allowances remain available. They cost about USD 29.49 with only the GKE credit and one current-size image pair; retained image history can add storage. This page owns those figures; 6.6. Platform Delivery, which plans the lab, links here instead of restating them.

Two cloud-side bounds keep that figure low:

• One small single-replica lab replaces a production HA topology.
• The GKE overlay right-sizes idle CPU requests for the two-core node while preserving each workload's burst limit.
• Artifact Registry deletes tagged or untagged image versions older than 30 days while preserving the five most recent versions. Versions younger than 30 days still accumulate, so this cleanup is not a storage cap.

The note below has the full arithmetic, including which billable SKUs — separately priced Google Cloud line items — you still pay for without the credit.

How do you prevent an idle bill?

Prefer local development. The Kubernetes control plane, node, disks, registry, and bucket keep billing while the cloud lab sits idle.

What proves this page worked?

For one representative trace, record:

1. Model calls.
2. Input/output tokens — read agentops.tokens.session.total off the span, or query agentops_tokens_token_total from host Prometheus at :9090.
3. Total latency.
4. Model path.
5. Current price source and date.

Confirm the budget trips by setting AGENT_MAX_TOKENS_PER_SESSION low and observing TOKEN_BUDGET_EXHAUSTED. For the GKE plan, list every billable resource and the free-tier assumption. Mark unknown values as unknown; do not claim a monthly total from machine type alone.

You are done when:

• agentops_tokens_token_total comes back non-empty from Prometheus for a turn you just sent.
• That turn's MLflow trace carries agentops.tokens.session.total and agentops.cost.session.estimate.
• With both per-1k prices set to a rate you can cite by page and date, a later turn reports a non-zero agentops.cost.session.estimate that matches your own hand calculation.
• With AGENT_MAX_TOKENS_PER_SESSION=1, the second turn of a session returns error_code="TOKEN_BUDGET_EXHAUSTED" and no model call is made.
• Your recorded numbers name a price source and a date, with every unknown value written as unknown.

Continue to 7.4. Feedback when you can point at the token count for one of your own sessions and say where the number came from.